Tagging and Parsing an Artificial Language An Annotated Web-Corpus of Esperanto

In the first half of this paper, we present and evaluate EspGram a Constraint Grammar (CG) -based parser for the artificial language Esperanto. The second half of the paper describes the compilation and annotation of a corpus of 18.5 million words covering Esperanto literature, news text and web pages. As a planned language, conceived to be easy to learn and flexible to use, Esperanto has a highly regular morphology, where clearly perceived morphemes match linguistic categories almost one-on-one. Also, the core lexicon of the language was designed to avoid unnecessary ambiguity. Thus, morphological/lexematic ambiguity is almost entirely restricted to cross-compound ambiguity, and the average number of morphological readings is 1.12 readings per non-name word, as opposed to around 2.0 for most natural languages (depending on the way ambiguity is counted). Though since its inception (Zamenhof, 1887), the language has been allowed to evolve as a living system, most changes have occurred at the lexical level, and the morphological system remains largely unchanged. On the other hand, the relatively free word order of the language in combination with syntactic usage influence from different natural languages has led to a language system and a speaker community very tolerant of syntactic variation, where norms are statistical rather than absolute. This situation has important bearings on both parsing technology and corpus linguistics. First, with a reduced need for disambiguation, a part-of-speech tagger can be assumed to be almost identical to a morphological analizer, while a syntactic parser will face a number of challenges. Second, a corpus of correct but international Esperanto may offer interesting insights in lexical and syntactic variation, reminiscent of the variation of non-native, international English, the difference being that in Esperanto, such variation is not stigmatized, but rather allowed or even supported by the flexibility of the language system.